Adaptive Attention Link-based Regularization for Vision Transformers

Although transformer networks are recently employed in various vision tasks with outperforming performance, extensive training data and a lengthy training time are required to train a model to disregard an inductive bias. Using trainable links between the channel-wise spatial attention of a pre-trained Convolutional Neural Network (CNN) and the attention head of Vision Transformers (ViT), we present a regularization technique to improve the training efficiency of ViT. The trainable links are referred to as the attention augmentation module, which is trained simultaneously with ViT, boosting the training of ViT and allowing it to avoid the overfitting issue caused by a lack of data. From the trained attention augmentation module, we can extract the relevant relationship between each CNN activation map and each ViT attention head, and based on this, we also propose an advanced attention augmentation module. Consequently, even with a small amount of data, the suggested method considerably improves the performance of ViT while achieving faster convergence during training.Comment: 14 pages, 5 figure

A globally exponentially stable position observer for interior permanent magnet synchronous motors

The design of a position observer for the interior permanent magnet synchronous motor is a challenging problem that, in spite of many research efforts, remained open for a long time. In this paper we present the first globally exponentially convergent solution to it, assuming that the saliency is not too large. As expected in all observer tasks, a persistency of excitation condition is imposed. Conditions on the operation of the motor, under which it is verified, are given. In particular, it is shown that at rotor standstill---when the system is not observable---it is possible to inject a probing signal to enforce the persistent excitation condition. {The high performance of the proposed observer, in standstill and high speed regions, is verified by extensive series of test-runs on an experimental setup

Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification

In person re-identification (ReID) task, because of its shortage of trainable dataset, it is common to utilize fine-tuning method using a classification network pre-trained on a large dataset. However, it is relatively difficult to sufficiently fine-tune the low-level layers of the network due to the gradient vanishing problem. In this work, we propose a novel fine-tuning strategy that allows low-level layers to be sufficiently trained by rolling back the weights of high-level layers to their initial pre-trained weights. Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks. The improved performance of the proposed strategy is validated via several experiments. Furthermore, without any add-ons such as pose estimation or segmentation, our strategy exhibits state-of-the-art performance using only vanilla deep convolutional neural network architecture.Comment: Accepted to AAAI 201

An Algebraic Approach to Quantum Systems Using Finite Group Representation

Thesis advisor: Dubi KelmerUndergraduate physics emphasizes the Schrödinger's analytic approach in solving and understanding quantum systems. Although briefly mentioned, group-theoretic approach is not emphasized, at least in terms of mathematical rigour. I attempt to kill the two rabbits - rigour and application. Part I develops the necessary formal theory of representation of finite group. In part II: Application, I primarily focus on the behaviour of an electron in various potentials where the spherical symmetry of an atom is broken into finite symmetry where I can readily apply the machineries that I have developed in part I. Basic notions of group theory, linear algebra and quantum mechanics are assumed.Thesis (BS) — Boston College, 2014.Submitted to: Boston College. College of Arts and Sciences.Discipline: Mathematics Honors Program.Discipline: College Honors Program.Discipline: Mathematics

Weighted Fourier analysis and dispersive equations

The goal of this thesis is to apply the theory of multilinear weighted Fourier estimates to nonlinear dispersive equations in order to tackle problems in regularity, well-posedness, and pointwise convergence of solutions. Dispersion of waves is a ubiquitous physical phenomenon that arises, among others, from problems in shallow-water propagation, nonlinear optics, quantum mechanics, and plasma physics. A natural tool for understanding the related physics is to study waves/signals simultaneously from both physical and spectral perspectives. Specifically, we will treat nonlinearities as multilinear operator perturbations, which (by the method of spacetime Fourier transforms), exhibit smoothing properties in norms defined to reflect the dispersive natures of the solutions. Our model equation is the quantum Zakharov system, which can be viewed as a variation on the cubic nonlinear Schrödinger equation (NLS). We investigate the model in various contexts (adiabatic limits, nonlinear Schrödinger limits, semi-classical limits). We additionally study a variation of Carleson's Fourier convergence problem in the context of pointwise convergence of the full Schrödinger operator with non-zero potential

Oxidase-Coupled Amperometric Glucose and Lactate Sensors with Integrated Electrochemical Actuation System

Unpredictable baseline drift and sensitivity degradation during continuous use are two of the most significant problems of biosensors including the amperometric glucose and lactate sensors. Therefore, the capability of on-demand in situ calibration/diagnosis of biochemical sensors is indispensable for reliable long-term monitoring with minimum attendance. Another limitation of oxidase enzyme-based biosensors is the dependence of enzyme activity on the background oxygen concentration in sample solution. In order to address these issues, the electrolytic generation of oxygen and hydrogen bubbles were utilized 1) to overcome the background oxygen dependence of glucose and lactate sensors and 2) to demonstrate the feasibility of in situ self-calibration of the proposed glucose and lactate sensors. Experimental data assure that the proposed techniques effectively establish the zero calibration value and significantly improve the measurement sensitivity and dynamic range in both glucose and lactate sensors

The Use of Multi-Scale Fiducial Markers To Aid Takeoff and Landing Navigation by Rotorcraft

This paper quantifies the impact of adverse environmental conditions on the detection of fiducial markers (i.e., artificial landmarks) by color cameras mounted on rotorcraft. We restrict our attention to square markers with a black-and-white pattern of grid cells that can be nested to allow detection at multiple scales. These markers have the potential to enhance the reliability of precision takeoff and landing at vertiports by flying vehicles in urban settings. Prior work has shown, in particular, that these markers can be detected with high precision (i.e., few false positives) and high recall (i.e., few false negatives). However, most of this prior work has been based on image sequences collected indoors with hand-held cameras. Our work is based on image sequences collected outdoors with cameras mounted on a quadrotor during semi-autonomous takeoff and landing operations under adverse environmental conditions that include variations in temperature, illumination, wind speed, humidity, visibility, and precipitation. In addition to precision and recall, performance measures include continuity, availability, robustness, resiliency, and coverage volume. We release both our dataset and the code we used for analysis to the public as open source.Comment: Extended abstract accepted at the 2024 AIAA SciTec

Two-dimensional covalent triazine framework as an ultrathin-film nanoporous membrane for desalination

We computationally demonstrate that two-dimensional covalent triazine frameworks (CTFs) provide opportunities in water desalination. By varying the chemical building blocks, the pore structure, chemistry, and membrane performance can be designed, leading to two orders of magnitude higher water permeability than polyamide membranes while maintaining excellent ability to reject salts.Netherlands Organization for Scientific ResearchUnited States. Dept. of Energy (Contract No. DE-AC02-05CH11231)Deshpande Center for Technological Innovatio

Glucose Oxidase (GOD)-Coupled Amperometric Microsensor with Integrated Electrochemical Actuation System

Recent developments for biosensors have been mainly focused on miniaturization and exploratory use of new materials. It should be emphasized that the absence of a novel in-situ self-calibration/diagnosis technique that is not connected to an external apparatus is a key obstacle to the realization of a biosensor for continuous use with minimum attendance. In order to address this issue, a novel solid-state glucose oxidase-coupled amperometric biosensor with integrated electrochemical actuation system has been designed and validated. There are two key components of the proposed glucose biosensor: solid-state GOD-coupled thin-lm amperometric sensing element and O2 depleting/saturating built-in electrochemical actuator. The actuator can be used to accomplish in-situ 1-point self-calibration by depleting O2 (i.e., by simulating glucose-free environment). Also, it can be used at the same time to extend the proposed sensor\u27s linear detection range by in ating O2 (i.e., by enhancing glucose sensitivity). A prototype sensor was fabricated and a series of lab experiments was conducted. Collected data assures that the proposed sensor effectively establishes the zero calibration point and signi cantly enhances its measurement sensitivity and con dence